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ABSTRACT 

This study examines the multi-modal synthesis of 
source examples in student compositions. Multimedia is used to 
present on-screen text and video information on Chinese superstitions 
and traditions, and then to prompt subjects to write. The research 
compares the number and kinds of examples students produce in their 
writing. Kind refers to the source of influence, either audio/video 
or the written transcript of that audio/video. Forty subjects from 
Washington State University participated in the study. Students were 
first asked to read a sample of professional writing and then to 
write a response to it. In addition, full motion laser audio/video 
was used as source material. The results demonstrated that the 
audio/video mode of presentation provides a significantly richer 
resource for detail than on-screen text for student compositions in 
the immediate task, but the audio/video resource does not sustain 
influence in the delayed composition task. These results provide 
support for the argument that i.nplementat i on of multimedia in 
composition instruction merits greater attention to task as well as 
to the interpretation and analysis of audio/video material if the 
rich audio/video resource potential multimedia makes available is to 
be useful in composition instruction. (Contains 17 references.) 
(Author/MAS) 
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Abstract: This study examines the multi-modal synthesis of source examples in student 
compositions. Multimedia is used to present on-screen text and video information on Chinese 
superstitions and traditions and then to prompt subjects to write. The research compares the 
number and kinds of examples students produce in their writing. Kind refers to the source of 
influence, either audio/video or the written transcript of that audio/video. The results 
demonstrated that the audio/video mode of presentation provides a significantly richer 
resource for detail than on-screen text for student compositions in the immediate task, but the 
audio/video resource does not sustain influence in the delayed composition task. These 
results provide support for the argument that implementation of multimedia in composition 
instruction merits greater attention to task as well as to the interpretation and analysis of 
audio/video material if the rich audio/video resource potential multimedia makes available is 
to be useful in composition insuiiction. 
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Introduction and Problem Statement 

As multimedia delivery systems for instruction have proliferated, they have found their way into 
writing intensive courses. The designs as well as the assumptions underlying the goals of integrating 
multimedia into education vary widely, but almost all suggest, at least in those courses where the outcome of 
instruction depends upon student writing, that multimedia, and particularly audio/video sources, have influenced 
that writing. The nature of that influence, however, has not been examined. Meanwhile, popular consensus 
points repeatedly and suspiciously toward the trend away from the primacy of print. As Maxson contends, 
"The influences of MTV exert more influence on student text than does the academy" (1993). 

Maxson's contention is not unfounded. Ninety-eight percent of American households own a television 
(Gates, 1993). As Gates observes, "It's MTV more than Montaigne, Viacom more than Voltaire" (Gates, 1993, 
p. 1 16). Much television research tends to support both the dominance of audio/video media consumption, and 
suspicions toward audio/video media's influence upon learning. Contrary to the more positive assumptions 
underlying multimedia's rapid development and implementation in the classroom, extensive consumption of 
visual media beyond a threshold of only ten hours per week begins to correlate with weak language skills, a 
correlation that is pronounced at viewing of over thirty hours per week (Potter, 1987). Graber (1990) has found 
that we learn very little from watching television news. Meanwhile, our apparent growing reliance upon visual 
media continues, encouraged by the availability and increasing ease of video manipulations with instantly 
digitized graphic capabilities now migrating into mainstream word processing programs. The decline of the 
primacy of the age of print, as McLuhan (1969) forecast, appears to be further accelerated by advanced memory 
compression and on line-video libraries, not to mention the implications of the rapid development and 
decreasing cost of hand-held camcorders, video boards, scanners, speech fluid word-processors and even virtual 
reality. All of these breakthroughs portend at least the transformation if not the demise of composition skills as 
we have known them. 

In spite of the perceived dangers of audio/video as it is represented by television, theories and limited 
studies claim that multimedia can be used to improve beginning student reading, writing, and general literacy 
skills beyond the scope of conventional instructional methodologies. The work of DiPardo and DiPardo (1990), 
Slatin (1990), Bransford et al. (1990) implicitly and overtly counter that media design can be used to encourage 



97 



BEST COPY AVAILABLE 



and improve the academic performance of students. For instance, Bransford's work argues that there is "more to 
notice" (1990, p. 124) in visuals than in text. Bransford argues that we can improve the academic performance 
of those students who tend to rely more on visuals, if, as Graber (1990) also cautions, the visuals are 
appropriately stimulating and adequately mediated. Still, even the enthusiastic multimedia practitioners 
acknowledge the difficulty involved in verifying their theories. As John Slatin says, multimedia technologies 
and writing instruction will require "both a new practice and a new rhetoric, a new body of theory" (1990). 



Source Tracking 

As multimedia provides an opportunity to mediate visuals, so source synthesis may provide one useful 
operationalization of multimedia influence upon composition. Research in writing from sources preceding 
multimedia technologies reflects that need to explore source synthesis, or the sources writers draw upon when 
they compose. - After all, expert writers, research has established, process, select, organize and connect source 
texts (Hayes et al., 1992; Spivey, 1992). As Spivey notes in her introduction to "Discourse Synthesis: Creating 
Texts from Texts" (1992), although students who worked from the same texts "produced texts that had some 
commonalties in content as well as in overall organizational patterns, the writers varied in how they chunked the 
material, how much material they selected as relevant, and how they connected the material for the reader" (p. 
469). Spivey maintains that such "discourse synthesis" is "characterized by its selectivity and its 
integrativeness" (p. 469). Such selection or choice is particularly important, Spivey maintains, because it must 
also be integrated to "provide coherence to material from diverse sources and perspectives" (p. 469). This 
"meaning construction," she argues, "includes both production and comprehension" (p. 470). Though Spivey's 
work deals exclusively with text, multimedia technologies make the need to explore discourse synthesis more 
acute. 

Concrete Detail 

Since multimedia provides an image rich environment, imagery suggests a natural focus for source 
investigation. After all, the use of images in text hns itself held a kind of primacy predating electronic media: 
"A picture is worth a thousand words." This aphorism has been held as truth even, ironically, in text. As 
Richard Ohmann observes, citing E. D. Hirsch, Jr. and his "accumulated wisdom of the handbooks," There are 
only two "maxims" that appear out of a dozen in every major handbook-- "Use definite, specific, concrete 
language" (1988, p. 353). Ohmann pursues these maxims in detail and into three additional popular handbooks 
that treat the issue extensively and cite the need to use specific details as a way to "avoid abstract writing" (p. 
354) and to make writing "'richer,' more 'vivid,' and more 'intense'" (p. 357). Does multimedia imagery directly 
influence the transformation of that imagery by students into their own texts? Is what we view as writers as 
accessible as what we read. The following study was designed to explore multimedia discourse synthesis to 
begin to assess the influence of multimedia, and specifically audio/video influence upon student writing. 

Methodology 

The writing assignments examined in this study were designed to reflect a common academic writing 
task in which students are first asked to read a sample of professional writing and then to write a response to 
that reading. In this study, however, in addition to exposing subjects to text, full motion laser audio/video was 
also used as source material for subjects before they were asked to write. The primary dependent measure in 
both the immediate and the delayed composition tasks was details used in subject's writing and, specifically, 
the source of that detail as it could be traced to the mode of the message, either the text mode or the audio/video 
mode presented in the multimedia environment. Secondary dependent measures included recall and time on 
source. 

Forty subjects from Washington State University participated in the study. Four treatments were 
sequenced to control for message mode and order. Each of the four treatments began with an introductory 
excerpt from Kingston's "No Name Woman," the subject matter for the second assigned written part of the 
study. Following the text excerpt from the Kingston source text, each treatment presented one of two possible 
audio/video supplements on Chinese traditions and superstitions and one of two possible on-screen text 
transcript supplements on Chinese traditions and superstitions. The text supplements were transcribed from the 
narrative of the video/narration supplements. The audio/video screens included controls for subjects to pause 
and review material in order to parallel the recursive nature of reading. 
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Apparatus, Materials, Stimulus and Facilities 

Hardware: 

The computer used to present the multimedia treatments was a Macintosh Ilci. The laser video player 
used was a Pioneer 8000 laser disc player. The video and narration were presented using a 19" Mitsubishi 
color television. 
Source Material: 

The laser program used as text supplement was the National Geographic ™video. "Portrait of a Hong 
Kong Family. " The program was selected based upon its depiction of themes parallel to those developed in the 
Kingston essay-superstition and traditions in China and their impact upon sex roles. The program was easily 
edited and re purposed to support and augment the themes in Kingston's essay/story. 

Maxine Hong Kingston's essay/story, "No Name Woman" is excerpted from her novel of the same 
name in a textbook, Writing About the World . 
Multimedia Design: 

HyperCard™ and The Voyager Video Stack'"''^ were used to create a linear stack, sequencing the 
screens subjects saw in each treatment from the first to the last without options. This design was selected 
because issues of multimedia navigation might have created distracting variables for the purposes of this study, 
though navigation issues, including providing the subject with control of the mode of delivery, certainly suggest 
future research. The last of the screens subjects viewed instructed them to write about Chinese tradition and 
superstition. The prompt encouraged subjects to use concrete examples and to write about a page and half. 

Procedures 

When subjects finished the multimedia program, they were prompted to describe what they had seen, 
heard, and read. When they finished, they were presented with the assignment to read the Kingston story/essay, 
and they scheduled a time to return and write again about the reading. Within a week, each subject returned to 
write about the influence of superstition and tradition on the role of women China. 

Rating subject compositions required two-steps. First, two raters identified details. The cross check 
reliability was .79. The second step of the rating procedure determined source mode distribution. Two additional 
source raters maintained a reliability of .95. 

In addition, following the composition task subjects completed a multiple choice recall test to control 
for recall variables. A post hoc written debriefing was conducted to further illuminate the findings. 

Results 

Immediate Composition Detail Synthesis 

Both message order of source mode presentation and recall were eliminated as a significant variables. 
It was therefore possible to collapse across order to increase the statistical power. (Instead of analyzing ten 
cases in each treatment, it was possible to statistically examine forty cases by each mode.) After collapsing 
across order, the design for subsequent analyses was a 2 (mode) x 2 (message) mixed subjects design. 

The mean for detail in subjects' writing traceable to the audio/video source was 15.9. The mean for 
detail traceable to text was 5.1 (E(l,39) = 51.67, p<.0001). The multimedia audio/video source significantly 
dominated the text source as an influence in the production of detail in subjects' writing. (See figure 1.) 

18 -\ • — p 

16 - I 

U - 

^12 - 

|10- 

5 e - 



2 - 

0 J-J 1 1 1 , 

C«11 

Figure 1. Immediate Composition Source Detail by Mode 
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Delayed Composition Detail Synthesis 

In the delayed composition task, there was no traceable evidence of the multimedia source, either 
audio/video or text. Since there was zero variance in either the on-screen text or the audio/video variables, a t- 
test was used to test the main effect of treatment mode to detail production. The effect is significant (t- Value = 
10.848, p<.0001). The means for detail traceable to the Kingston source text at 16 was significantly higher 
than the means for audio/video or on-screen text-both of which were 0. (See Figure 2). 
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Figure 2: Delayed Composition Source Detail by Mode 
Discussion 

The dominant influence of the audio/video in the immediate composition task, in spite of the sequence 
of presentation, supports the contention that multimedia and particularly audio/video "mini-cases" can be rich 
in information or detail (Spiro and Nix, 1992; Bransford, 1992) and that "the visual material was more 'potent"' 
than text (Martin & Ditcham, 1987, p. 10). At least during the immediate composition task, students were more 
than three times as likely to use detail presented on the television screen than they were to use detail presented 
as text on a computer screen. 

Contrary to the audio/video dominance in the immediate composition task, however, the delayed 
composition task shows no evidence of multimedia influence— audio/video or text. The potential use of 
audio/video established earlier in this study was supplanted entirely by source text dominance. All of the 
traceable detail used in the delayed composition task was synthesized from either personal experience (X=6) or 
the BCingston story, "No Name Woman" (X=16). Since both time on source and recall were controlled for, this 
finding poses a number of interesting questions for further research. 

Perhaps the sequence of the design, a design in which students read the Kingston text after 
experiencing the multimedia treatment, results in recency effects that wash any recall of the multimedia sources. 
However, the complete absence of traceable multimedia influence coupled with the debriefing explanations 
substantially subordinate or even eliminate recency or other related effects. If the traceable multimedia sources 
were diminished in the delayed composition task, recency might provide an important explanation. But the few 
subjects who do refer to the tendency to write about the most recent stimulus tend to qualify their explanations 
in ways that suggest that it is not an inability to recall multimedia sources that influenced subjects' composition 
decisions. For instance, in post experimental debriefings, one subject, who described the delayed composition 
aspect of the experiment as "a typical [freshman English] assignment. . . one that requires time and recollection 
of facts," said: "It never occurred to me to use the information from the previous day in the essay I wrote today. 
I tend to focus more on the task at hand and what's given to me in regards to the task. It's really strange to me 
that I didn't even think to use the previous information I received." Several additional debriefing comments 
corroborated this student's observation. 

It may be, of course, that the absence of multimedia source influence simply reflects that the prompt 
was not specifically clear in suggesting the possibility for using multimedia source material. Grimes explains 
conflicting findings in his work by pointing to "the nature. . . of the task itself and experimenter instructions. . ." 
(in press, p. 42), But the assignment did clearly prompt for descriptive detail of Chinese superstition and 
tradition, stating: "Discuss in as much detail as you can . . . ." In other words, more than simply a failure of the 
assignment to sufficiently prompt students to draw upon the multimedia sources, it seems more likely that 
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subjects for some reason do not perceive the "typical" composition assignment as a task that might 
accommodate material from previously presented multimedia sources. 

Perhaps, the contrary findings between the immediate and delayed composition tasks may best be 
understood by considering the following debriefing response. The subject explains: "I consciously and 
purposefully drew only upon specific and general examples set up by the reading. Unconsciously, I think the 

video lent background and insight into my writing The video gave me a background on the importance of 

such superstition, but I don't think that it caused me to analyze it any further. In writing the essay, I think I felt 
obligated or bound to writing about the text." 

First of all, notice again that the subject reports memory for the video source. Secondly, though, notice 
that the subject reports of a perception of the task in which he feels "obligated or bound to writing about the 
text." One post hoc emerging finding, in other words, was that task representation was instrumental in 
influencing the kind of information student subjects developed in the delayed composition task. In this case, 
though, like the subject who felt "obligated" to write exclusively about the text, the perception of the assigned 
task prohibited the transfer of the previously presented multimedia material-text screen or audio/video. Task, 
as Flower and Hayes (1988) define it, involves the mental representations that shape our goals, our concept of 
audience, and the construction of a text itself: "People only solve the problem they give themselves to solve" (p. 
^.^^•./^^ "image of one's rhetorical.problem . . ." (1988, p. 93) as Flower and Hayes contend, has been 
significantly influenced by the relationship of previous academic experience, multimedia or otherwise, to the 
delayed composition task, in this case a task whose representaiion has been shaped by much more than just the 
message mode. As one subject reports, "In my writing. . . I want to hit everything relevant to what I think one 
wants from my writing" (Italics mine). Who the subject here regards as audience is not clear, but who or what 
It IS often elicits a sense of obligation that significantly shapes student writing and the synthesis of detail in a 
way that significantly favors the most recently encountered traditional text. 

The findings in the delayed composition study reported here, then, suggest the need to regard task as 
more than simply an artifact of message mode. Nonetheless, there is also reason to explore the relationship of 
message mode to the perception of task and not to dismiss it as insignificant. For instance, consider again the 
subject who said, "The video gave me a background on the importance of such superstition, but I don't think that 
It caused me to analyze it any further . . ." . Ironically, it seems prudent not to completely disregard the 
message mode itself, in this case the "image" of the video. Is there, perhaps, an aspect of the video mode that 
elicits a resistance in subjects to "analyze" the video information "any further," something inherent in the 
audio/video message mode that reshapes, or fails to shape, "the image of one's rhetorical problem"? 

Within the context of this study there is substantial fodder for speculation relative to multimedia 
implementation and multimedia discourse synthesis. As Spiro et al contend: "A fundamental tenet of all recent 
theories of comprehension, problem solving, and decision making is that success in such cognitive arenas 
depends on the activation and appropriate application of relevant preexisting knowledge" (1986, p. 177). Surely 
It is reasonable to include composition and multimedia discourse synthesis as aspects of "comprehension," 
"problem solving" and cognitive "decision making." Consequently, it may be wise in future research to 
consider both the conceptualization of task as well as the conceptual transfer of mode specific "relevant 
preexisting knowledge" in terms of "appropriate application" constraints. 

Study of discourse synthesis writing as Spivey (1992) practices and advocates, and multimedia source 
synthesis, as Palumbo and Prater (1993) predict, are valuable approaches toward understanding the composition 
process. The findings in this study may begin to be useful in putting together the inscrutable puzzle of that 
process and in the design and delivery of multimedia environments. Clearly audio/video information provides a 
rich source of detail that is available to writers during the composing process. But it appears equally important 
to understand that there is nothing magical about multimedia sources that will inform the traditional textual 
analysis without more control of the implementation of the multimedia than this study, a study based largely 
upon current academic practices, afforded. 

Finally, and extending from the previous point, even if the transfer of discreet detailed information 
from multimedia e.nvironments can be harnessed, there is no evidence that quantity of detail alone will improve 
student writing. In other words, the detail dominance of the audio/video mode apparent in the immediate 
composition task does not necessary correlate with higher quality writing. In that sense, as critics fear, new 
media harbors as much threat as promise if it is not re-purposed toward more cognitively complex tasks that 
include transfer and analysis. The point to point digital nature of audio/video sources, or the "Being There" 
effects of audio/video sources, provide a great deal of potential for education as well as entertainment but the 
pitfalls loom large. 

More than simply the mode of the message, it remains the implementation or mediation of the media 
that seems to really matter. It is a complex task. It will continue to require teachers. 
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